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ABSTRACT 

Over the past decade, public attention on the importance of the civic development and education 
of youth has grown. To address these concerns, the East Bay Conservation Corps (EBCC) Charter School 
opened in 1996 with the explicit mission to prepare and engage students grades K through 12 as caring 
citizens who are capable and motivated to fully participate in our democracy. While content standards 
and assessments readily exist to articulate the academic and artistic development of students, youth civic 
development, especially at the elementary level, has been under-conceptualized. What is needed is a 
more robust, comprehensive developmental framework for citizenship education that begins with younger 
ages and addresses civic skills and dispositions to the same degree as civic knowledge. 

The product from this project is a set of tested, reliable measures of civic knowledge, civic 
thinking skills, civic participation skills and civic dispositions that are referenced to recent efforts to 
provide frameworks of competencies in civic education. Two sets of instruments were developed using a 
comprehensive conceptual framework for civic indicators at the elementary level. The measures include 
a student survey of student civic knowledge, skills and attitudes that relate to dispositions, which is 
the focus of this report; a set of corresponding grade level observation checklists of student skills and 
behaviors was also developed. 

Starting at a young age to foster developmental foundations for civic engagement includes a 
democratic orientation to others and identification with them as fellow members of a community and 
body politic. This focus is not only developmentally appropriate but also consistent with the goals of 
many elementary schools to foster prosocial skills and behaviors. In addition, there is a need for greater 
attention to age-appropriate, instrument identification and development for elementary aged students to 
document student civic development by focusing on what they can do, an important and often overlooked 
facet of K-12 civic education research and practice. Addressing this need will also assist other public 
elementary schools interested in recapturing their civic mission and in creating a K-12 developmental 
framework for civic development. 
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SECTION I: BACKGROUND 

Over the past decade, public attention 
on the importance of the civic development and 
education of youth has grown due to patterns 
of perceived youth civic disengagement, 
including declining voting rates among 18-24 
year olds, low interest in political participation 
and deliberation, and gaps in knowledge about 
fundamental democratic principles and processes 
(Carnegie Corporation and CIRCLE, 2003; Niemi 
and Junn, 1998; Putnam, 1996, 2000). As a 
result, increasing numbers of foundations, state 
legislatures and organizations have focused 
attention on how to promote civic engagement 
and prepare youth for their role as active and 
responsible citizens. To address these concerns, 
the East Bay Conservation Corps (EBCC) Charter 
School opened in 1996 with the explicit mission to 
prepare and engage students grades K through 12 
as caring citizens who are capable and motivated to 
fully participate in our democracy. 

Since 1983, the EBCC has provided 
leadership in serving low-income urban youth and 
in developing active learning strategies that imbue 
young people with a sense of their role in the 
community. The EBCC Charter School includes two 
divisions. The Corpsmember High School opened 
in September of 1996 and is focused on meeting 
the immediate educational and employment needs 
of students between the ages of 17 to 24 years. 

The Elementary Level of the EBCC Charter School, 
the focus of this study, opened in September of 
2001 and was created out of the belief that public 
schools must prepare children for the challenges, 
opportunities and responsibilities of life in a 
democratic, pluralistic society. The EBCC Charter 
School strives to instill in students three kinds of 
literacy: 

Academic Literacy: The ability to 
read, write, speak, calculate, reason, 
and conduct processes of inquiry 
with clarity and precision. 

Artistic Literacy: The ability to learn 
and creatively express oneself 
through the visual, performing, 
literary arts, and technology, and 



Civic Literacy: The ability to "let 
your life speak" by participating 
thoughtfully, responsibly, and 
passionately in the life of the 
community with concern for the 
common good. 

The curriculum framework and overall 
design for the school was created through a 
three year planning process by a national team of 
educators representing all grade levels and from 
various fields of expertise, including civic education, 
spiritual development, research and evaluation, 
and educational policy. When looking for guidance 
in planning the school, however, we encountered 
many limitations to the existing civic education 
literature that are well summarized by Torney-Purta 
and Vermeer (2004). 

Most standards and assessments of civic 
development focus on civic knowledge 
with significantly less attention paid to 
civic skill building or the formation of civic 
dispositions. Many lists of citizenship 
competencies and standards frequently 
consist of encyclopedic coverage of details 
of government structures or historical 
documents that may have little meaning to 
students and do not connect to their own 
identity as a citizen with responsibilities and 
rights or to their motivation to learn about 
their communities. The desired outcomes 
are often complex, making it difficult to 
adapt them for students in the early years 
of elementary school or for immigrants 
and/or second-language learners. They 
also typically cover the same topic at 
several grades (such as the founding of the 
United States) rather than cumulatively 
building more complex understanding 
based on earlier basic concepts. Finally, 
the topics tend to focus primarily on 
patriotic observances that are important 
but insufficient as preparation for engaged 
citizenship (Torney-Purta and Vermeer, 

2004, p. 1). 
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Because of these limits within the existing 
citizenship education literature, we experienced 
the need to articulate an appropriately broad 
framework for citizenship development and civic 
engagement at the elementary level that fit with 
the EBCC Charter School mission. In addition, we 
were unable to find developmental^ appropriate 
instruments for students of this age that would 
help us document progress and challenges in 
implementing this broader civic framework at 
the elementary level in an urban setting. As a 
result, in partnership with Abt Associates and 
Brandeis University, the East Bay Conservation 
Corps proposed this project to address one of 
CIRCLE'S priority areas to identify developmental^ 
appropriate indicators to assess progress in 
civic education at the elementary grades. With 
a national team of civic education and service- 
learning experts, we developed and piloted test 
measures of civic outcomes appropriate for 
younger school students. The product from this 
project is a set of tested, reliable measures of 
civic knowledge, skills, dispositions and behaviors 
that are referenced to recent efforts to provide 
frameworks of competencies in civic education. 

The focus of this report is to summarize 
the process of developing and pilot testing the 
measures, including the reliability and validity 
of the measures. Two sets of instruments were 
developed using a comprehensive conceptual 
framework for civic indicators at the elementary 
level that included civic knowledge (what students 
should know about citizenship), civic thinking skills 
(cognitive civic skills students should posses), civic 
participation skills (participatory civic skills students 
should possess) and civic dispositions (civic 
dispositions, students should possess), expanding 
the framework used in Torney-Purta & Vermeer, 
2004 and adapting it for use in this setting. The 
measures include a student survey of student 
civic knowledge, skills and attitudes that relate 
to dispositions, which is the focus of this report; 
we also developed a set of corresponding grade 
level observation checklists of student skills and 
behaviors. (See Appendix A for the Student Survey 
used in the national pilot and Appendix B for the 
Student Observation Checklists.) 



The rest of this report is organized in the 
following way: Section I (Background) continues 
with an overview of the issue of civic development 
at the elementary level and the proposed project to 
develop assessments. The needs for measures not 
addressed by existing instruments as well as the 
challenges faced when assessing civic development 
at the elementary level are also addressed. 

Section II (Methodology) describes the project 
methodology in more detail and Section III (Design 
of Instrument) outlines the instrument design 
process, including the conceptual frameworks and 
items that were identified or adapted for use in 
the student survey. Section IV (Results) presents 
evidence of the reliability and validity of the 
student survey as well as findings from the student 
survey. Section V (Conclusion and Implications) 
provides discussion of the results and implications 
for this work, including proposed next steps to 
continue to improve the measures and support civic 
development in the elementary grades. 



OVERVIEW OF ISSUE AND PROJECT 

Much of the discussion and research on 
youth civic engagement and civic education 
is focused on high school aged youth, college 
students and young adults (e.g. Andolina, Keeter, 
Zukin and Jenkins, 2003; Eyler and Giles, 1999; 
Morgan and Streb, 2001; National Association of 
Secretaries of State, 1999; Perry and Katula, 2001; 
Walt Whitman Center for the Culture and Politics 
of Democracy, 1998). Between ninth grade and 
college graduation, there are substantial gains in 
political knowledge and civic skills but the starting 
point is not a total lack of knowledge or unformed 
attitudes. For example, studies suggest that young 
children demonstrate awareness of social issues 
and exhibit understanding of democratic decision- 
making processes, democratic principles such as 
rights and freedoms, and concepts of fairness 
(Berman, 1997; Berti, 2005; Helwig, 1998; Helwig 
and Jasiobedzka, 2001; Hess and Torney, 1967; 
Moore, Lare and Wagner, 1985). 

Studies of elementary and middle school 
students also suggest that the average student in 
democratic countries is already a member of his 
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or her political culture by the end of elementary 
school. For example, the students' trust in 
government-related institutions already match in 
many respects those of adults in their society (Hess 
and Torney, 1967; Torney-Purta and Vermeer, 
2004). Because the early grades represent a 
critical opportunity to lay a foundation upon which 
civic knowledge, skills and dispositions can grow, 
it is important to examine what is appropriate 
to expect of elementary grade level students 
regarding civic outcomes such as knowledge, 
dispositions, skills and behaviors. 

However, there is some disagreement 
among civic education researchers about whether 
explicit civic education should start at the 
elementary level. It has been suggested that the 
cognitive development required by civic education 
is not well suited to or developmentally appropriate 
for five to ten year olds. This sole emphasis on 
civic education as teaching civic content, however, 
actually limits a more robust, comprehensive 
understanding of civic education as the teaching 
of civic knowledge, skills and dispositions. As 
a result, civic education at the elementary level 
remains seriously underconceptualized and has 
meant that there has been limited attention paid to 
a developmental framework for civic education that 
extends from grade K-12. This project reframed 
this issue in a different way: What does civic 
development look like at the elementary grades? 
And how do we measure it in a way that will 
stimulate reflection on the part of researchers and 
educators about directions that are appropriate in 
this area? 

Although civic education research may not 
focus on the elementary grades, other research in 
areas relevant to civic education have found fertile 
ground in this age range. For example, social 
and emotional development, moral development, 
character development, and conflict resolution/ 
violence prevention programs are all relevant to 
the development of civic skills and dispositions 
and are considered developmentally appropriate 
for elementary grade students (e.g. Flannery et 
al., 2003; Lickona, 1991; Turiel, 1983; Watson et 
al., 1989). For researchers and practitioners in 
civic education at the elementary level, it would 



be helpful to view such work in the context of a 
broader civic education framework. 

At the same time, the practice of civic 
education at the elementary level has far outpaced 
the support provided by research, and such 
research deserves more attention. For example, 
CityYear, a full-time national service program 
targeting 17-25 year olds in 17 cities across the 
country, initiated an effort in 2003 to create a Civic 
Index to track the civic awareness, motivation, 
capacity, identity and actions of elementary 
and middle school students tutored by City 
Year corpsmembers. The Constitutional Rights 
Foundation has continued to expand its work in 
Teaching American History at elementary schools. 
The Association for Supervision and Curriculum 
Development in partnership with The First 
Amendment Center created the First Amendments 
Schools (FAS) project in 2001 to help elementary, 
middle and high schools affirm First Amendment 
principles and put them into action throughout their 
communities. 

In addition, there is a substantial need for 
more research on civic development for youth 
in urban areas who face particular challenges to 
civic engagement (e.g. Hart and Atkins, 2002) 
and for youth from immigrant, racial and ethnic 
minorities (Junn, 2004; Sanchez-Jankowski, 

2002). For example, there is evidence that youth 
from immigrant or racial/ethnic minorities may 
define civic engagement in different ways that 
do not fit traditional measures or understandings 
of civic engagement and education. While in- 
depth exploration of these populations are not 
the primary focus of this study and such further 
study is needed, the expanded framework of civic 
development beyond traditional civic education 
that is focused on teaching content and the 
resulting instruments for assessment represent 
important contributions to the field of civic 
education research and practice. Although the 
development of the instruments occurred as a 
result of our work in an elementary school with 
diverse student demographics in a distinctly urban 
environment, our desire to expand the concept of 
civic development at the elementary level to value 
content, skills and dispositions clearly resonated 
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with educators in rural, suburban and other urban 
communities who work with diverse student 
populations. 

NEED FOR MEASUREMENT NOT FILLED BY OTHER 
INSTRUMENTS: 

One of the biggest challenges faced by 
educators of the EBCC Charter School in 2001 
when launching the elementary school was 
the dearth of tested measures to assess civic 
development outcomes of elementary school 
students, particularly their civic attitudes, skills 
and dispositions. While content standards and 
assessments are readily available to articulate the 
academic and artistic development of students, 
youth civic development, especially at the 
elementary level, was understood in a rather vague 
way. Our staff spent a considerable amount of time 
developing a set of school wide expectations that 
constituted a broad framework for civic literacy 
of what we expected students to know and be 
able to do. In brief, it includes components of 
personal responsibility, caring for others and for 
the community, and leadership to take positive 
actions. A recent California Department of 
Education document articulating the relationship 
between service-learning and civic responsibility 
accurately captures our ultimate goal of creating 
caring, capable citizens who fully participate in our 
democratic society: 

Individuals participate and engage 
in the affairs of their community 
in a variety of ways and along a 
continuum of commitment. This 
continuum begins with commitment 
at the personal level, through 
individual volunteerism and 
community service, for example, 
and extends to active participation 
in civic affairs in ways that promote 
systemic change by addressing 
deeply rooted issues of public 
policy. Each level of commitment 
is beneficial to the community and 
serves to empower youths both 
individually and collectively, and 
as such, serves the greater good. 

Civic education seeks to encourage 



youths to participate across the 
continuum of commitment, but 
especially to identify public systems 
that inadequately serve the common 
good and work to change them. 

(California Department of Education, 

2003, p.5) 

In addition to our own search for curriculum 
and assessment, through an inquiry of other civic 
education programs at the elementary level, we 
were not the only ones struggling with these 
issues. In fact, when hearing about this project, 
many of those schools and programs volunteered 
to pilot test our measure, demonstrating the 
need for such instruments. While a number of 
substantial measures exist for middle school, high 
school and post-secondary students (e.g. Andolina, 
Keeter, Zukin and Jenkins, 2003; Kahne and 
Middaugh, 2005; Torney-Purta, Lehmann, Oswald & 
Schulz, 2001; Walt Whitman Center for the Culture 
and Politics of Democracy, 1998), we did not 
identify comparable measures for younger students 
through a review of the literature, although we 
did find a few instruments that addressed some 
aspects of our multidimensional framework of civic 
development that included knowledge, skills and 
dispositions. 

For example, items from the National 
Assessment for Educational Progress (NAEP) 
were heavily focused on civic content 
which we acknowledge is an important 
component of civic education. Such content 
is not, however, the only priority for civic 
development at the elementary grades 
especially given other interests in students' 
prosocial and emotional development at that 
age level. Also, NAEP items are released 
for use on a very limited basis (though a 
number of screened items from NAEP were 
released on the web at www.ecs.org/qna in 
2005). As noted earlier, the national service 
program, CityYear, created a Civic Index that 
included constructs such as civic awareness, 
motivation, capacity, identity and actions in 
addition to aspects of youth development 
and academic learning. Their assessment, 
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however, did not include components of 
civic skills or knowledge and thus did not fit 
our framework of civic outcomes that drew 
from current work in civic development and 
education. 

When we expanded our review of 
assessments beyond those self-described as 
civic or citizenship development to include 
what we considered important components 
of civic education (such as certain skills 
or attitudes), we found more measures 
from fields such as social and emotional 
development, character development, 
and school climate assessments. For 
example, web-based databases exist for 
social and emotional research compiled by 
organizations such as the Center for Social 
and Emotional Education (www.csee.net) 
and The Collaborative for Academic, Social 
and Emotional Learning (www.casel.org) with 
assessments relevant for the elementary 
grades. A similar database and assessment 
index exists for the character education 
field (www.character.org) with research 
and instruments that focus on elementary 
aged students and younger. CART, or the 
Compendium of Assessment and Research 
Tools, has also compiled a list of assessment 
tools for youth development programs 
(www.cart.rmcdenver.com) although there 
were few measures for elementary aged 
students. Another widely recommended set 
of measures to assess school climate and 
sense of community that are relevant to the 
social, emotional and moral development 
of elementary aged students were those 
developed for the Child Development Project 
by the Developmental Studies Center (Watson 
et al., 1989). Several of their constructs 
focused on measuring social competence, 
conflict resolution, caring for others and 
democratic values. 

Reviewing examples of these 
instruments was useful to capture the 
language appropriate to elementary school 
students and to examine the potential 
connections to civic development, but 



these assessments were limited in scope 
for educators interested in capturing the 
broader and multi-dimensional nature of 
civic development within an instrument. As 
a result, this project addressed an important 
need in the research and practice of youth 
civic engagement, especially since at the 
time we developed our proposal and our 
instrument, the Education Commission of the 
States had not yet incorporated extensive 
screened civic disposition items in their web 
site (www.ecs.org/qna). 

The EBCC Charter School presented an 
ideal environment to develop these measures 
given its educational philosophy, curriculum and 
careful selection of teachers and other staff who 
are knowledgeable about and committed to the 
academic, artistic and civic development of its 
students. It also represented an opportunity 
to develop measures within a multicultural, 
socioeconomic and religiously diverse student 
population in an urban area. A glimpse of the 
elementary school's 2004-05 enrollment data 
paints the picture of a very diverse student 
population. 22% of the students/families classify 
themselves as multi-ethnic, 54% African American, 
14% white, 7% Hispanic, 1% Pacific Islander, and 
2% Asian. Students and families of the school 
are also linguistically diverse with home language 
backgrounds that include Spanish, Punjabi, 
Japanese and Burmese. The school is also socio- 
economically diverse with 39% of all families 
eligible for free lunch and 22% eligible for reduced- 
priced lunches with the remaining 39% of families 
exceeding the qualifying level for free and reduced 
lunches. These numbers alone, however, do not 
accurately describe our unique population. For 
example, in family household surveys, we have 
found that many of our students come from homes 
where the overall socio-economic level is low but 
where the average education level of the child's 
caregiver(s) is quite high (e.g. two or more years 
of college). 

This unique demographic terrain, which 
we believe is an outcome of the EBCC's multi- 
dimensional and dynamic urban context, is both a 
challenge and, we believe, a perfect opportunity to 
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study the development of an active citizenry. The 
intent of this project, however, was not only to 
support the school in its assessment and evaluation 
efforts, but also to contribute its products to 
the larger civic education fields of research and 
practice. 

CHALLENGES OF ASSESSMENT OF CIVIC 
DEVELOPMENT AT THE ELEMENTARY LEVEL 

Developing assessments for civic 
development at the elementary level presents 
several challenges faced by civic education 
researchers and practitioners interested in this 
age group. For this project, one set of challenges 
concerned the choice of the assessment format. 

For example, student self-reported assessments 
through paper-pencil measures with Likert or 
numbered scales are most efficient in capturing 
large amounts of data. Based on a review of 
existing civic education programs at the elementary 
level, however, it was determined that a reliable 
student survey using a relatively simple-to-use 
Likert scale (from 1 to 4) was needed to reflect 
a broader conception of civic development at the 
elementary level. The issue of uneven reading 
abilities at the elementary level, however, 
precluded the use of this instrument with children 
younger than third grade. So although it was 
limited to upper elementary grades, we felt it would 
be an important contribution to the fields of civic 
education research and practice. 

In addition, to provide teachers of students 
in kindergarten through second grade with an 
assessment tool that was consistent with our 
conceptual frameworks, we developed student 
observation checklists of students' skills and 
behaviors. Teachers at the EBCC Charter School 
found them useful in documenting civic literacy 
throughout the year, but they were not the focus 
of this study. At this point, they remain a teacher- 
focused tool that can be adapted to any classroom 
to document skills and behaviors that are relevant 
to civic development. Clearly, there is much more 
work to be conducted to verify the validity and 
reliability of the observation checklists, as will be 
discussed in the final section. 

The second set of challenges was 
conceptual in nature. In elementary grades, 



there is a tendency to award "good citizenship" 
grades based on obedience to classroom and 
school rules and demonstration of good work 
habits (neat handwriting, homework completion, 
etc.). Yet other conceptions of citizenship and 
civic engagement also exist and ultimately may 
be deemed desirable, such as active participation 
in one's community or a principled position from 
which individuals question unjust rules, laws or 
circumstances. (See Kahne and Westheimer, 2003 
for a brief discussion of various conceptions of 
citizenship.) As a result, we felt a need to frame 
civic development at the elementary level that 
connected these seemingly conflicting views of 
citizenship to prepare all of our students for the full 
spectrum of civic commitment and engagement 
opportunities. 

In creating our civic development framework 
for the elementary grades, we decided that with 
younger children, it was important to build a 
foundation of democratic knowledge, skills and 
dispositions that will enable students to understand 
what it means to be part of a larger group or 
community. In addition to promoting prosocial 
skills that could be viewed as civic participation 
skills, part of that knowledge and skill set is to 
take responsibility for oneself and one's actions; to 
realize that consequences come with one's choices; 
to recognize the effect that individual actions and 
choices have on larger groups; and to understand 
the need for rules that are in effect for the safety 
and welfare of the larger group. 

With this foundation in place, as children 
mature, they would be better able to make 
thoughtful choices that may question classroom, 
school and society's rules. They would make 
such choices with a greater understanding and 
awareness of the consequences of their choices 
and with reasoning to back up their choices. 
Children as young as five are able to question 
and make judgments (for example, when they 
deem a situation as "not fair"). They are not 
able, however, to articulate reasons why certain 
decisions or choices should be made which is more 
developmental^ appropriate in older grades. 

In short, at the elementary level, the 
developmental foundations for civic engagement 
is a democratic orientation to others and an 
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identification with them as fellow members of the 
body politic - so listening, respecting others, being 
responsible for one's own actions, is what happens 
in the elementary grades. Our ultimate goal is 
to create a K-12 framework that encourages and 
prepares our students to participate across the 
continuum of civic commitment, but especially to 
identify public systems that inadequately serve 
the common good and to work to change them 
(California Department of Education, 2003). 

SECTION II: METHODOLOGY 

To guide this project and provide 
constructive feedback on its products, we 
convened a distinguished advisory team of civic 
education and youth development researchers and 
practitioners: 

• JoAnn Jastrzab, Principal Research 
Associate, Abt Associates, Cambridge, 
Massachusetts 

• Alan Melchior, Deputy Director and Senior 
Research Associate at the Center for Youth 
and Communities, Brandeis University, 
Waltham, Massachusetts 

• Connie Flanagan, Professor, Penn State 
University, University Park, Pennsylvania 

• Joseph Kahne, Professor and Director 
of the Institute for Civic Leadership, Mills 
College, Oakland, California 

• Mary McFarland, Past President of the 
National Council of the Social Studies and 
Education Consultant, Chesterfield, Missouri. 

• Yolanda Peeks, Director of Professional 
Development Services, Developmental 
Studies Center, Oakland, California 

• Judith Torney-Purta, Professor, University 
of Maryland, College Park, Maryland 



The project launched with a visit of the advisory 
team to the EBCC Charter School to allow the team 



to observe the teachers and students in order to 
develop a common understanding about which 
civic outcomes may be reasonably expected in the 
elementary-aged children. The site visit included 
a working session involving the advisory team 
members and EBCC Charter School teachers and 
administrators to identify constructs associated 
with civic education in elementary school. 

A conceptual framework was developed and 
revised based on feedback from the advisory 
team using constructs of civic literacy (Personal 
Responsibility, Civic Responsibility, and Leadership) 
that were originally developed by the faculty and 
staff at the EBCC Charter School as important 
components of civic development at the elementary 
level. (The conceptual frameworks will be 
discussed in more detail in Section III.) Following 
the visit, the advisory team refined the measures 
through regular email correspondence and 
continued to monitor current research to identify 
any relevant measures that have been validated as 
part of other studies. 

A. INITIAL PILOT 

The first pilot version consisted of two versions 
of a student survey that contained different scales 
and items to maximize our opportunity to pilot the 
widest range of items. One version (37 items) 
was administered to the fourth grade class (n=20) 
of the EBCC Charter School in June of 2005. The 
other version (38 items) was administered to 
the fifth grade class (n = 19) at the same time. 

The students spent approximately 20 minutes 
answering the questions. Items or words that 
caused students to raise their hands and ask 
questions during the administration of the survey 
were noted for revision. Four or five students also 
participated in focus groups after completing the 
survey to identify any items which they considered 
unclear or otherwise problematic. Questions 
asked during the focus groups included "what did 
you think about the survey?", "which words were 
confusing to you?" and "which questions were 
hard for you to answer?" Students' observations 
about the survey ranged from "it was boring" and 
"easy" to "you should have asked more questions 
about what we think about academics" to "it was 
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interesting" and "I liked answering the questions." 

B. SECOND PILOT (REVISION AND NATIONAL 

PILOT PHASE) 

Based on feedback from the initial pilot and 
comments from our advisory team, the survey 
was shortened to four pages, including one page 
of survey instructions and student information. 
Items were revised or eliminated based on student 
questions about confusing items or words. In 
addition, based on factor analysis and Cronbach's 
coefficient alpha results for each of the constructs 
in the student survey, more items were eliminated 
to increase the reliability of the constructs. For 
example, in many cases the negatively stated items 
were difficult for students of this age to answer. 

A national sample of 550 elementary and 
middle school students were recruited to test 
the reliability of the measures from elementary 
schools involved in the following organizations and 
networks, many of which the EBCC has established 
collaborations and partnerships: Education 
Commission of the States, Corporation for National 
and Community Service Learn and Serve America 
schools, national AmeriCorps programs working 
in schools such as City Year, National Service- 
Learning Partnership, National Council for the 
Social Studies, the First Amendment Schools, 
California Department of Education CalServe 
Office, Constitutional Rights Foundation and Youth 
Service California among others. We obtained 
written parental consent in advance of any testing 
of students and those consent forms reside at the 
local sites. 

What was affirming about the schools 
that decided to participate was that principals 
and teachers first reviewed the survey and 
then volunteered to participate in the study 
because they found value in the survey. In 
total, 40 teachers from 18 schools in seven 
states participated in the national pilot, including 
students from Alaska, New Flampshire, California, 
Massachusetts, Kansas, Indiana and Minnesota. 

The primary goal was to seek the 
participation of upper elementary grade students 
(fourth or fifth grades). 12 third graders (2.2%) 
participated in the survey. 175 fourth graders 



(32.3%) and 256 fifth graders (46.9%) constituted 
the bulk of the data. 46 sixth graders (8.4%) also 
participated in the pilot and in one site, a district 
administrator also included 56 seventh and eighth 
grade students (10.3%). 

The national sample of convenience was 
very diverse. Although 16.4% of the students 
declined to state their race or ethnicity, of those 
students who did, white students constituted 
35.7% of the sample, with the next largest group 
representing multiracial or other groups (22.4%). 
Latino students represented 19.8% of the national 
sample, with African American (8.3%), Asian/ 
Pacific Islander (9.1%) and American Indian (4.3) 
students constituting the remainder of the sample. 
The sample was somewhat skewed toward females 
(57.5%) versus males (42.5%). 

Most of the classrooms reported that they 
involved their students in some aspect of civic 
education, whether it was teaching American 
history within a specific classroom or involvement 
in a school-wide or district-wide initiative to 
promote character development, civic education, 
and/or service-learning. 



SECTION III: DESIGN OF INSTRUMENT 
A. DEFINITION OF VARIABLES TO BE 
MEASURED: 

To guide this project, we initially 
developed a conceptual framework 
that corresponded to the different 
levels of students: (1) kindergarten 
and first grade; (2) second and 
third grade; and (3) fourth and fifth 
grade. The framework for each 
grade span included the following 
civic components deemed important 
and relevant to foster in elementary 
grades: 

(1) Personal Responsibility: The student 
demonstrates responsible behaviors and 
good judgment and accepts responsibility 
for one's own behavior. The student also 
demonstrates responsible work habits such 
as staying on task, working independently 



www.civicyouth.org 



10 






CIRCLE Working Paper 47: June 2006 Developing Indicators and Measures of Civic Outcomes for Elementary School Students 



and showing best effort. 

(2) Caring for Others and Community: The 
student shows courtesy and respect for 
others and finds ways to help others. The 
student also shows respect for and is able to 
identify needs and solutions for group and 
community. 

(3) Leadership: The student takes initiative 
and acts as role model to help group, class 
or school to make a positive difference. 

In addition, each grade-specific framework consists 
of Civic Knowledge, Civic Thinking Skills, Civic 
Participation Skills and Civic Dispositions that are 
critical to the development of civic literacy, drawing 
from the format of the assessment database 
being complied by staff of the National Center 
for Learning and Citizenship at the Education 
Commission of the States (see Figures 1 - 3). 

These concepts and skills were developed with 
input from the EBCC Charter School teachers' 
understanding of what "civic literacy" and 
"citizenship" looked like at each grade level. In 
addition, civic education experts served as advisors 
to this project and the following documents 
served as resources that informed the content 
and format of the frameworks: Developing 
Citizenship Competencies Kindergarten through 
Grade 12 (Torney-Purta and Vermeer, National 
Center for Learning and Citizenship - Education 
Commission of the States), History--Sociai Science 
Content Standards for California Public Schools 
(California Department of Education), Education 
for Democracy: California Civic Education Scope 
and Sequence (Center for Civic Education/Los 
Angeles County Office of Education), Creating 
Effective Citizens (National Council for the Social 
Studies) and the National Standards for Civics and 
Government (Center for Civic Education). 
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Figure 1. K-l THEME: Taking Responsibility for Self and Fostering Awareness of Others 



Schoolwide Expectations for Civic Knowledge Civic Thinking Skills 

Civic Literacy What students should know Cognitive civic skills students 

about citizenship should possess 



Personal Responsibility 

Reponsible Behavior 

• Uses good judgment and 
accepts responsibility for 
own behavior 

Work Habits 

• Stays on task, works 
independently and shows 
best effort 

Caring for Others and 
Community 

Concern for Others 

• Shows courtesy and respect 
for others and finds ways 
to help others 

Concern for Group or 
Community 

• Shows respect for and is 
able to identify needs and 
solutions for group and 
community 

Leadership 

Takes initiative and acts as 
role model to help group, 
class or school to make a 
positive difference 



Know: 

What a “community” is and 
means (such as family, 
classroom, school, church, 
YMCA, etc.) 

What it means to be an active, 
responsible citizen or 
community member by 
participating in positive ways 
in the community 

Reasons for rules and laws 

Likeness and difference in our 
pluralistic society-- - cultural 
groups, customs, holidays, 
etc. 



Be able to: 

Recognize individual 
uniqueness as well as 
similarities and differences 
among people (starting with 
themselves and expanding to 
others) 

Show awareness of other 
people’s thoughts, feelings 
and/or experiences 

To think and talk about what 
s/he is doing and why 

Share one’s own opinions 
after gathering information 
and providing supporting 
evidence 



Civic Participation Skills 

Participatory civic skills 
students should possess 

Be able to: 

Manage one’s own behavior 
and develop the ability to 
comply with positive “group 



Communicate with others 
including listening to others 
and appropriately expressing 
one’s views 

Demonstrate group 
membership skills such as 
sharing, taking turns and 
being considerate of others, 
voting on class issues, and 
abiding by decisions arrived 
at by consensus or vote 

Cooperate with peers and 
resolve conflict without 
fighting - use grade level 
appropriate language of 
conflict resolution such as “I” 
messages 



Civic Dispositions 

Civic dispositions students 
should possess 

Demonstrate: 

Respect oneself and exercise 
appropriate independence, 
such as expressing one’s 
needs, taking care of personal 
hygiene, tying shoes, etc. 

Willingness to focus on given 
tasks within a classroom 
environment 

Willingness to follow 
norms/rules and “live with” 
the consequences of their 
behavior 

Respect others, others’ 
belongings, respect animals, 
and the environment 

Courtesy and empathy for the 
feelings of others 

Willingness to take action to 
help others and to provide 
service to the classroom or 
school community 
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Figure 2. SECOND/THIRD GRADE THEME: Helping Others and Our Community 



Schoolwide Expectations for 
Civic Literacy 


Civic Knowledge 


Civic Thinking Skills Civic Participation Skills 


Civic Dispositions 


What students should know 
about citizenship 

Know: 


Cognitive civic skills students 
should possess 

Be able to: 


Participatory civic skills 
students should possess 

Be able to: 


Civic dispositions students 
should possess 

Demonstrate: 


Personal Responsibility 

Reponsible Behavior 

• Uses good judgment and 
accepts responsibility for 
own behavior 

Work Habits 

• Stays on task, works 
independently and shows 
best effort 

Caring for Others and 

Community 

Concern for Others 

• Shows courtesy and respect 
for others and finds ways 
to help others 

Concern for Group or 

Community 

• Shows respect for and is 
able to identify needs and 
solutions for group and 
community 

Leadership 

• Takes initiative and acts as 
role model to help group, 
class or school to make a 
positive difference 


What is a larger community - 
such as neighborhood and city 
and what it means to be part 
of a community 

What it means to be an active, 
responsible citizen or 
community member 

Leaders in a community and 
what they do — how decisions 
get made, who makes them 
and how they affect us 

Importance of rules and laws 
and how they are created and 
changed 

Relationship of past to present 

History of diverse cultures in 
the local community and our 
pluralistic democracy 


Compare community life long 
ago with life today 

Demonstrate problem solving 
and analytical skills by 
identifying issues and needs, 
studying to build knowledge, 
and proposing possible 
solutions 

Gather information and begin 
to generalize and transfer 
understanding to other 
experiences 

Use basic decision making 
process for addressing needs 
in neighborhood 
organizations and city 
government 

Take the perspective of others 
to understand how people feel 
and what they intend by what 
they do 


Participate in designing, 
implementing and analyzing 
community needs assessment 

Give input on possible 
solutions to larger community 
(such as neighborhood and 
city) needs 

Initiate contact, hold a 
conversation with and 
demonstrate the ability to 
build a relationship with a 
community member 

Engage in thoughtful civil 
dialogue about important 
personal, school, and 
community issues 

Cooperate with peers and 
helps others avoid and settle 
conflicts 


Willingness to balance group 
needs with personal needs 

Interest in taking action to 
address needs and issues in 
the larger community 

Responsibility and caring for 
animals, plants, people and 
environment in larger 
community 

Recognize the authority of 
people and institutions in the 
local community and 
recognize the opportunity for 
change and growth 

Ability to reflect on a process 
and gain insight for future 
experiences 

Respectful exposure to 
diverse opinions and people 
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Figure 3. FOURTH/FIFTH GRADE TFIEME: Taking Leadership to Make a Positive Difference 



Schoolwide Expectations for 
Civic Literacy 


Civic Knowledge 


Civic Thinking Skills 


Civic Participation Skills 


Civic Dispositions 


What students should know 
about citizenship 

Know: 


Cognitive civic skills students 
should possess 

Be able to: 


Participatory civic skills 
students should possess 

Be able to: 


Civic dispositions students 
should possess 

Demonstrate: 


Personal Responsibility 

Reponsible Behavior 

• Uses good judgment and 
accepts responsibility for 
own behavior 

Work Habits 

• Stays on task, works 
independently and shows 
best effort 

Caring for Others and 
Community 

Concern for Others 

• Shows courtesy and respect 
for others and finds ways 
to help others 

Concern for Group or 
Community 

• Shows respect for and is 
able to identify needs and 
solutions for group and 
community 

Leadership 

Takes initiative and acts as 
role model to help group, 
class or school to make a 
positive difference 


What it means to be an active, 
responsible citizen or 
community member 

What is a larger community - 
such as neighborhood, city 
and state and what it means to 
be part of a community 

Basic understanding of 
leaders, structures, functions 
and powers of local, state and 
federal governments and how 
they impact local issues 

Democratic values such as 
equality, justice, rights and 
responsibilities 

Why it is important to 
participate in a democracy 

History of social issues and 
social change movements 


Analyze problems and think 
through potential solutions, 
consequences and obstacles 

Demonstrate research skills 
such as finding multiple 
sources, analyzing validity of 
information, drawing 
conclusions and presenting 
findings 

Use basic decision making 
process for addressing needs 
in neighborhood 
organizations and city 
government 

Take the perspective of others 
to understand how people feel 
and what they intend by what 
they do 


Analyze community issues 
and needs and propose and 
work on possible solutions 

Demonstrate communication 
skills by listening carefully to 
others and using appropriate 
language to accurately 
express one’s ideas and 
opinions 

Listen to different ideas in a 
group setting and work with 
others to formulate a 
suggestion that works for the 
group 

Engage in thoughtful civil 
dialogue about important 
personal, school, and 
community issues 

Cooperate with peers and 
helps others avoid and settle 
conflicts 

Apply knowledge to propose 
and implement civic action 


Sense of responsibility and 
initiative to take action 

Commitment to balance self- 
interest with interest in the 
common good such as 
concern for future generations 

Responsibility and caring for 
animals, plants, people and 
environment in larger 
community 

Willingness to consider 
multiple perspectives and to 
participate in discussion to 
build consensus 

Respect for diverse opinions 
and people 

Critical thinking about 
institutions and government 
in the larger state and national 
community 
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IDENTIFICATION OF ITEMS 

Items for the student survey were 
identified or adapted from existing instruments. 
Instruments reviewed for appropriate items that 
fit our conceptual framework for civic development 
included the following: Child Development Project 
(Developmental Studies Center), CityYear Civic 
Youth Development Index Survey (CityYear), 
California Civic Index (Kahne and Middaugh/ 

Mills College), KIDS Consortium Student Survey 
(Melchior/Brandeis University), Four-Fold Youth 
Development Model (Purdue University), Civic 
Responsibility Survey (Furco et al. /University of 
California, Berkeley), among others. In addition, 
new items were created through collaboration with 
the EBCC elementary school teachers. 

LIST OF CONSTRUCTS AND ITEMS IN NATIONAL 
PILOT 

The items selected for the national pilot 
are categorized by scale below. The numbers in 
parentheses represent the item number on the 
survey. We used a Likert scale from 1 to 4 which 
was consistent with other measures for elementary 
school children. The responses ranged from 1 
(Strongly Disagree or "No Way!"), 2 (Disagree or 
"Not really"), 3 (Agree or "Sort of") and 4 (Strongly 
Agree or "Yes!"). 

Personal Responsibility: This construct was 
defined as demonstrating responsible behaviors 
and work habits. 

Personally Responsible Behaviors - Moral 

1. If I break something, I try to fix it. (Item 
# 1 ) 

2. I put things away when I am done with 
them. (Item #22) 

3. I always try to do my best work. (Item #25) 

4. If I do not do a good job, I try to do better 
the next time. (Item #31) 

Personally Responsible Behaviors - 
Conventional 

5. I usually do what I'm supposed to do. (Item 
# 12 ) 

6. It is important for me to follow the rules 



even if no one is watching. (Item #19) 

7. I think it is important for people to follow 
the rules. (Item #38) 

Civic Responsibility: This construct was 
defined as caring for others, valuing group 
work, caring for community, appreciating 
diversity and demonstrating environmental 
stewardship. 

Concern for Others 

8. I try to help when I see people in need. 
(Item #4) 

9. When I make a decision, I try to think about 
how other people will be affected. (Item 
# 10 ) 

10. 1 try to be kind to other people. (Item #11) 

11. 1 apologize when I hurt someone's feelings. 
(Item #20) 

12. 1 want to help when I see someone having a 
problem. (Item #46) 

Value of Group Work 

13. To solve most problems, I have to learn how 
to work with others. (Item #2) 

14. 1 can learn more from working on group 
projects than from working alone. (Item 
#16) 

15. 1 like working with other people on group 
projects. (Item #24) 

Caring for Community 

16. 1 spend time on projects with other people 
to help the community. (Item #23) 

17. 1 think it is important to change things that 
are unfair in society. (Item #27) 

18. 1 have done things to help people in my 
community. (Item #37) 

19. 1 believe that I can make a difference in my 
community. (Item #42) 

Appreciating Diversity 

20. 1 want to have friends who have different 
backgrounds from me. (Item #14) 

21. 1 can learn a lot from people with 
backgrounds and experiences that are 
different from mine. (Item #43) 
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Environmental Stewardship 

22. 1 try to get my family to recycle at home. 
(Item #6) 

23. 1 have a responsibility to help keep the 
community clean. (Item #17) 

24. 1 try to get my friends to recycle bottles and 
cans. (Item #39) 

25. 1 do my part to help the environment. 

(Item #47) 

Leadership Efficacy: This construct was defined 
as demonstrating the attitudes and actions of 
taking leadership positions. 

26. Once I know what needs to be done, I am 
good at planning how to do it. (Item #7) 

27. When I see something that needs to be 
done, I try to get my friends to work on it 
with me. (Item #13) 

28.1 am pretty good at organizing a team of 
kids to do a project. (Item #18) 

29. If I'm the leader of a group, I make sure 
that everyone in the group feels important. 
(Item #21) 

30. 1 feel like I can stand up for what I think 
is right, even if my friends disagree. (Item 
#28) 

31. When I see something that is wrong, I try to 
change it. (Item #33) 

Civic Thinking Skills: This construct was defined 
as the ability to think critically. 

32. It is important for me to get information to 
support my opinions. (Item #5) 

33.1 am able to give reasons for my opinions. 
(Item #29) 

34.1 keep my mind open to different ideas 
when planning to make a decision. (Item 
#30) 

Civic Participation Skills: This construct 
was defined as skills in perspective taking, 
communication, group membership and conflict 
resolution. 



Perspective Taking Skills 

35.1 try to think how someone else would feel 
before I say something. (Item #3) 

36.1 make sure I understand what another 
person is saying before I respond. (Item 
#48) 

37. When I am listening to someone, I try to 
understand what they are feeling. (Item 
#49) 

Communication Skills 

38.1 try to think before I say something. (Item 
#9) 

39.1 summarize what another person said to 
make sure that I understood. (Item #36) 

40.1 try to watch other people's body language 
to help me understand what they are trying 
to say. (Item #41) 

Group Membership Skills 

41. When I play with others, I take turns. (Item 
#44) 

Conflict Resolution Skills 

42. If friends are fighting, I try to get them to 
talk to each other and stop fighting. (Item 
#8) 

43. If a friend is mad at me, I try to understand 
why. (Item #40) 

44.1 know how to avoid a fight when I need to. 
(Item #45) 

School as a Community: This scale was adapted 
from a much longer scale created by the 
Developmental Studies Center used to assess 
the sense of community that exists within 
school. 

45. Most students at this school treat each other 
with respect. (Item #15) 

46. Most students seem to care about each 
other, even people they do not know well. 
(Item #26) 

47. Teachers at this school won't let students 
make fun of other students. (Item #32) 

48. Students have an opportunity in this school 
to debate and discuss issues. (Item #34) 

49. Students feel like they are an important part 
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of the school. (Item #35) 




50. Students can talk to the teachers in this 






school about things that are bothering 
them. (Item #50) 




Civic Knowledge Items: Five items were selected to represent concepts and standards 


covered primarily in grades three through five. 




51. 


Which of these people make local laws? 
a. Police officers 


c. City council members 




b. Newspaper reporters 


d. Business leaders 


52. 


July 4 is a national holiday that celebrates the day when: 




a. the American colonies declared their 


c. women won the right to vote in elections. 




independence from Britain 
b. the Pilgrims arrived in the New World. 


d. the Civil War officially ended. 


53. 


“I pledge allegiance to the flag of the United States of America, and to the republic for which it 




stands, one nation under God, indivisible, with liberty and justice for all.” 




What are you promising when you say the pledge of allegiance? 




a. I promise to be loyal to the ideals of the 


c. I promise to salute the fag when I see it 




United States. 


in parades. 




b. I promise to study the laws of the United 


d. I promise to join the army of the United 




States. 


States. 


54. 


A student notices that the playing fields at her school have become littered with trash. How 




could she best show her civic responsibility? 
a. Complain to her friends that no one at the 


c. Organize her class or school club to 




school cares about the way it looks. 


clean up the playing fields. 




b. Ignore the mess and take her friends to 


d. Call the police to report that the 




play in an area that is still clean 


school’s fields are dirty. 


55. 


What is a good reason to have laws? 
a. To get people into trouble. 


c. To protect people. 




b. To help the president make money. 


d. To keep police busy. 
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The civic knowledge items were included to 
represent the comprehensive framework of civic 
knowledge, skills and dispositions within the 
survey and were also requested by several of 
the pilot sites. Given the significant attention 
focused on civic content in grades K through 12, 
the bulk of the development work for this project 
centered on the civic skills and dispositions that 
have received limited attention at the elementary 
level. As a result, the focus of this report is on the 
first 50 items of the survey. The following section 
describes the reliability and validity of the student 
scale. 

SECTION IV: RESULTS 

RELIABILITY: 

Reliability of the survey scales was 
measured generating Cronbach's alpha scores 
for each of the constructs. The results for the full 
national sample are illustrated in the following 
table: 

In general, when outlier items are removed, 



there were good reliabilities for most scales ranging 
from .64 to .78 with exception of two scales: 
Appreciating Diversity and Critical Thinking Skills. 

Several possible factors may account for 
the low reliability for Appreciating Diversity scale. 
The limited number of items included in this 
scale (only two items) is one that deserves more 
research; the addition of more items would likely 
increase its reliability. Also, there were problems 
reported by survey administrators and students 
about the language of the items. For example, 
students in several sites reported that the word 
"background" in the phrase "different backgrounds" 
was confusing or not familiar. This was not an 
issue in the pilot phase, possibly because such 
language may be more common in urban, diverse 
communities. As our nation's pluralistic democracy 
continues to diversify, this scale requires additional 
attention to assess students' understanding 
and appreciation of diverse backgrounds and 
experiences. 

The Critical Thinking Scale also 
demonstrated low reliability. Once again, this 



Scale 


# of Items in 
Scale (National 
Pilot) 


Internal 

Consistency 


Outlier Items 


Internal 
Consistency 
w/o Outliers 


Personally 

Responsible Behaviors 
(Originally two scales) 


4 


.42 (Moral) 




.68 

(Combined) 


3 


.63 

(Conventional) 




Concern for Others 


5 


.74 




.74 


Value of Group Work 


3 


.58 


Item #2 


.66 


Caring for Community 


4 


.69 


Item #27 


.72 


Appreciating Diversity 


2 


.49 




.49 


Environmental 

Stewardship 


4 


.71 




.71 


Leadership Efficacy 


6 


.64 




.64 


Critical Thinking 
Skills 


3 


.48 




.48 


Civic Participation 
Skills 


10 


.82 


Items #3 and #9 
caused this scale 
to split into two 
factors 


.78 


School as Community 


6 


.64 




.64 


Civic Knowledge 


5 


NA 


NA 


NA 


TOTAL SCALE 


55 
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may be due to the small number of items (only 
three items) currently in the scale. The addition of 
more items might increase reliability. In addition, 
although our teachers and students in the initial 
pilot phase did not report concerns about these 
items, several students in the national pilot sites 
found these items to be somewhat confusing, 
indicating that both the language and concepts 
in the scale may be challenging for this age 
group. Clearly, given the desirability to promote 
aspects of critical thinking as an important skill for 
citizenship, this scale warrants additional attention, 
including further clarification of the construct given 
the various connotations to the phrase, "critical 
thinking." That is, of what, if anything, do we 
expect students at this age to be thinking critically? 

A few other findings regarding the overall 
reliability of the scales should be noted. First, 
the Personally Responsible Behaviors scale had 
originally been organized into two separate 
scales to reflect the distinction between moral 
and conventional thinking that is considered 
developmental^ appropriate for this age group. 

This distinction also emerged during the findings 
from the initial pilot phase. However, based 
on factor analysis of the national sample data, 
these items did fit into one scale and combining 
the items from the original two separate scales 
into one improved its reliability significantly. We 
also acknowledge that a survey is not the best 
methodology to make fine distinctions such as this. 
As a result, we suqqest usinq it as a combined 



scale for Personally Responsible Behaviors. 

Second, based on factor analysis, the Civic 
Participation scale contained two items, Item 
#3 (" I try to think how someone else would feel 
before I sav something. "! and Item #9 ("I try to 
think before I say something/'), which caused 
the ten item scale to break into two factors. As 
a result, although removing those items caused 
the reliability to drop slightly from .82 to .78, we 
suggest removing those items to create a single 
scale and one factor. Finally, although we suggest 
removing Items #2 ("To solve most problems, I 
have to learn how to work with others.") and #27 
("I think itj-'s important to change things that are 
unfair in society. ") to increase the reliabilities 
of the scales, analyses of the reliability of the 
scales by racial and ethnic group revealed some 
interesting patterns that suggest the items could 
be useful for some subpopulations, as will be 
discussed in more detail in the next section. 

RELIABILITY OF SCALE WITH SUBPOPULATIONS 

We also tested the reliability of each scale 
using subpopulations of the national sample, by 
gender, by age and by racial/ethnic groups. These 
reliabilities are generated using data that removed 
the outlier or other perceived problematic items 
cited above (item #2, item #3, item #9 and item 
#27). 

When analyzing the results by gender, 
there did not appear to be major differences in 
the reliability of the scales. The two problematic 



Scale 


Internal Consistency by Racial/Ethnic Group 
(without Outlier Items #2, #3, #9 and #27) 


Boys 

(n=231) 


Girls 

(n=313) 


Personally Responsible 
Behaviors 


.72 


.61 


Concern for Others 


.76 


.65 


Value of Group Work 


.64 


.67 


Caring for Community 


.70 


.72 


Appreciating Diversity 


.49 


.50 


Environmental 

Stewardship 


.68 


.74 


Leadership Efficacy 


.68 


.59 


Critical Thinking Skills 


.46 


.50 


Civic Participation 
Skills 


.81 


.72 


School as Community 


.62 


.67 
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scales, Appreciating Diversity and Critical 
Thinking, remained unreliable for both groups. 

The Leadership Efficacy scale for girls was lower 
(.59) than for the national sample. However, its 
reliability was not improved by removing additional 
items which suggests that girls may have had 
issues with the items or the construct, a suggestion 
that may not be surprising to those familiar with 
gender analysis of leadership styles but this finding 
should be further explored. 

When analyzing the demographics of the 
national by grade, there were only 12 third graders 
and seven seventh graders in the national sample. 
As a result, to examine reliability of the scale by 
age, we excluded the third grade students and 
analyzed reliability of the scale for fourth grade 
(n = 176), fifth grade (n = 256) and sixth through 
eighth grades (n = 102) to constitute a middle 
school age population. 

Overall, the scales remained generally 
reliable across age groups from grades four 
through eight. The two scales, Appreciating 
Diversity and Critical Thinking Skills, continued 



to show low reliability, even as students mature, 
indicating that the scales are problematic and not 
an issue of reading level or comprehension. It 
is interesting to note that the reliability for the 
Value of Group Work scale increased slightly (from 
.66 to .68) for the middle grades when item #2 
was included. This suggests that the item is less 
confusing and perhaps more appropriate for this 
age group than for fourth or fifth grade students. 

Given the work of Junn (2004) and Sanchez- 
Jankowski (2002), it was viewed as worthwhile 
to assess the reliability of the scales by racial 
and ethnic group to see if there were differences 
among them. Unlike the reliability analyses of 
the other subgroups by gender or age, analyses 
by racial and ethnic group is mostly suggestive, 
especially for those smaller groups such as African 
American and Asian/Pacific Islander students. 
Because the number of Native American students 
was so small (n = 20), the results for that subgroup 
are not reported here and the survey should be 
administered with a larger number of students to 
test its reliability. Overall, these analyses should 



Scale 


Internal Consistency by Racial/Ethnic Group 
(without Outlier Items #2, #3, #9 and #27) 


Fourth Grade 
(n=176) 


Fifth Grade 
(n=256) 


Sixth-Eighth Grades 
(n=102) 


Personally 

Responsible Behaviors 


.70 


.68 


.65 


Concern for Others 


.69 


.77 


.71 


Value of Group Work 


.65 


.65 


.66 


Caring for Community 


.75 


.72 


.71 


Appreciating Diversity 


.48 


.46 


.57 


Environmental 

Stewardship 


.70 


.70 


.78 


Leadership Efficacy 


.68 


.64 


.61 


Critical Thinking 
Skills 


.45 


.56 


.37 


Civic Participation 
Skills 


.81 


.78 


.78 


School as Community 


.66 


.64 


.69 
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be viewed as exploratory in nature and should be 
the subject of further study. 

As illustrated by the table above, there 
was some variation in the reliability of the scales, 
with most of the scales demonstrating reliability 
across most groups with a few exceptions. The 
Concern for Others, Caring for Community and 



of the scale for African Americans. Had it been 
included, the reliability would have increased from 
.32 to .60 for African Americans but would have 
decreased the reliability of the scale from .63 to .47 
for Latino students, indicating the need to test the 
items and scale with larger numbers of students to 
ensure that the scale is still valid to use with those 



Scale 


Internal Consistency by Racial/Ethnic Groi 
(without Outlier Items #2, #3, #9 and #27) 


IP 

i 


White 

(n=164) 


Latino 

(n=91) 


African 

American 

(n=38) 


Asian/Pacific 

Islander 

(n=42) 


Other/Multi- 

ethnic 

(n=105) 


Personally Responsible 
Behaviors 


.79 


.49 


.70 


.62 


.70 


Concern for Others 


.76 


.79 


.66 


.75 


.71 


Value of Group Work 


.69 


.63 


.32 


.40 


.65 


Caring for Community 


.76 


.75 


.82 


.74 


.70 


Appreciating Diversity 


.26 


.56 


.18 


.60 


.50 


Environmental 

Stewardship 


.72 


.78 


.73 


.76 


.67 


Leadership Efficacy 


.60 


.73 


.45 


.71 


.64 


Critical Thinking 
Skills 


.51 


.57 


.16 


.43 


.53 


Civic Participation 
Skills 


.79 


.86 


.49 


.66 


.80 


School as Community 


.72 


.38 


.65 


.75 


.57 



Environmental Stewardship showed decent 
reliability (.60 to .82) across the groups. However, 
the Personally Responsible scale for Latino students 
dropped in reliability to .49. Interestingly, when 
Item #25 ("I always try to do my best work.") was 
excluded, the reliability of the scale went up to 
.60. In fact, the reliability of that scale increased 
for all subgroups, except for White students, when 
that item was removed. However, since the scale 
remained reliable for the other groups with that 
item included, it is suggested that the item remain 
within that scale but be carefully analyzed with 
samples including significant number of Latino 
populations. 

The reliability scores of the Value of 
Group Work scale excluded Item #2 ("To solve 
most problems, I have to learn how to work with 
others/') Excluding this item reduced the reliability 



subgroups. 

Although the reliabilities for Caring for 
Community were consistently above .60 across the 
subgroups, when Item #27 ("I think itj-s important 
to change things that are unfair in society.") was 
included in the scale, it increased the reliability of 
the scale in some groups (Latino and Other/Multi- 
Ethnic) and decreased the reliability of the scale in 
other groups (White, African American and Asian/ 
Pacific Islander). For the last two groups, the 
effect was to dramatically reduce the reliability of 
the scale from .74 to .48 for Asian/Pacific Islander 
students. 

Other scales such as Leadership Efficacy 
and Civic Participation also decreased below .60 for 
African American students, again suggesting the 
need to check the scales with larger populations 
of students. However, as indicated earlier, 
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these findings represent trends to be further 
explored given the small numbers of some of 
the subpopulations. The use of qualitative data 
collection such as interviews and focus groups 
would be particularly helpful to explore the 
potential reasons for these differences or issues 
with the items and scales. 

VALIDITY: 

Evidence for content validity of the student 
survey is strong in that the selected survey 
items were based on the conceptual frameworks 
described above; the instrument includes 
items from other measures of civic and social 
development; the instrument was reviewed for its 
content by teachers and administrators concerned 
and experienced with civic development and civic 
education at the elementary level; and the items 
were reviewed by a national advisory group of 
experts in service-learning, youth development and 
civic education. 

To collect evidence for construct validity of 
the student survey, we also examined correlations 
between the scales (see Appendix C). Overall, 
we would expect correlations to be fairly high 
among some of these scales as they all attempt 
to measure some component of civic development 
that holds concern for others and community 
as a valued attitude and motivation to use skills 
and demonstrate certain behaviors. However, 
theoretically, even with relatively high correlations 
of .50 to .60 among some of the scales, we still 
articulate each scale as a distinct component of 
civic knowledge, skills or dispositions. 

For example, the Civic Participation Skills 
scale was highly correlated with the most scales 
including Personal Responsibility (.653), Concern 
for Others (.684) and Leadership Efficacy (.613). It 
makes sense that students who feel a strong sense 
of personal responsibility and concern for others 
would report higher levels of civic participation 
skills as they would motivated by such attitudes to 
communicate and work well with others. However, 
the Civic Participation Skills scale measures the 
relative importance students place on certain skills 
versus attitudes. 

The following scales were distinct from 



all of the others with correlations of .50: Value 
of Group Work, Appreciating Diversity, Critical 
Thinking Skills and School as Community. Two of 
these scales (Appreciating Diversity and Critical 
Thinking Skills) were unreliable and so it is not 
surprising that the correlations would be low. We 
would also expect the School as Community scale 
to be distinct because it assesses students' school 
climate and does not represent a self-assessment 
of knowledge, skills or attitudes. It is interesting 
that the Value of Group Work is distinct from other 
scales, suggesting that it represent a relatively 
unique component of civic development that should 
be further explored. 

Given the relatively high correlations among 
some of the scales, two future activities would be 
helpful to confirm construct validity: confirmatory 
factor analysis (which will probably show correlated 
factor models) and an attempt to correlate specific 
aspects of the student observation check list to the 
corresponding parts of the student survey. 

To further examine evidence of construct 
validity, we also analyzed the survey results by 
gender and by racial and ethnic groups. Mean 
scores for girls were consistently higher than for 
boys in all of the scales, with eight of the eleven 
mean differences by scale being statistically 
significant at the .05 level (see Appendix D). Since 
many of the items reflected a helping orientation, 
this finding is consistent other studies that report 
a greater likelihood of girls or young women 
interested in volunteering and helping others (Hess 
and Torney, 1967; Jenkins, 2005). 

Finally, given the small number of student 
responses within some of the racial and ethnic 
subgroups, we examined findings by scale 
comparing the responses of students who were 
white and non-white. We also excluded students 
who identified as multi-ethnic or other as it 
was unclear how strongly they identified with a 
particular ethnic or racial group (see Appendix E). 
Based on this comparison, there were two scales 
with mean differences at the .05 level (Value of 
Group Work and Concern for Others). Interestingly, 
this data suggests that students of racial and 
ethnic minorities may value group work more than 
white students. The difference goes in the other 
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direction with the Concern for Others scale in that 
white students seem to value a more generalized 
concern for others more than non-white students 
who may be more concerned with the interests and 
welfare of their own ethnic group. These findings, 
while very exploratory, could support Sanchez- 
Jankowski's argument that white students and 
students from ethnic and racial minorities view civic 
engagement in different ways and also supports 
the constructive validity of the scales. In addition, 
white students slightly outperformed non-white 
students on the civic knowledge items. Although 
the difference was small, it was significant. This 
also confirms previous research that suggests 
that white students tend to outperform non-white 
students on tests of civic content (Niemi and Junn, 
1998). (For descriptive statistics of the full national 
sample, see Appendix F.) 

SECTION V: CONCLUSION AND IMPLICATIONS 

Based on the EBCC's work as a public 
elementary school with an intentional focus on 
creating capable, caring citizens as well as the 
interest demonstrated by the teachers in schools 
and districts who piloted this instrument, there is 
clearly a need for a research-based, comprehensive 
developmental framework for citizenship education 
that begins with younger ages and addresses civic 
skills and dispositions to the same degree as civic 
knowledge. Starting at a young age to foster 
developmental foundations for civic engagement 
includes a democratic orientation to others and 
identification with them as fellow members of a 
community and body politic. This focus is not 
only developmentally appropriate but consistent 
with the goals of many elementary schools to 
foster prosocial skills and behaviors. It is also an 
important and often overlooked facet of K-12 civic 
development and education. 

SUGGESTIONS FOR NEXT STEPS 

Our study found that it was not only feasible 
and appropriate to teach civics to elementary 
school students, it is also possible to measure the 
outcomes associated with this education. While 
this is a first step, there needs to be much more 
thinking both about what it means to learn civic 



skills and dispositions at an early age and how to 
assess them. This is an important contribution, 
but would benefit from a richer literature on 
both the teaching and assessment sides of the 
equation. Judging from the number of types of 
schools interested in this pilot, there is an unmet 
need to support elementary school educators and 
researchers in these areas. 

As a result we plan to disseminate 
a summary of this study and copies of the 
instruments to networks of researchers, 
practitioners and policy makers interested in 
civic education and development, including the 
Compendium of Assessment and Research Tools 
managed by RMC Research Corporation; Education 
Commission of the States; American Youth Policy 
Forum; Corporation for National and Community 
Service; National Council of the Social Studies; 
the First Amendment Schools; Association for 
Supervision and Curriculum Development; and 
the National Service-Learning Partnership. We 
also believe that interest will not be limited to 
school-based initiatives as many community based 
organizations such as 4-H, Scouts, Girls/Boys Clubs 
and others are also eager for assessments and so 
we plan to share our results with those networks as 
well. 

In particular, there are several possibilities 
for building upon and expanding this work, 
including: 

• refinement of the student survey instrument to 
improve the language and the addition of items 
that link the survey to other instruments for 
older students/youth; 

• additional data collection and analysis to 
improve evidence for the validity of instrument, 
including expanding, piloting and revising 
corresponding teacher rating checklists for use 
to collect evidence for criterion validity; 

• the collection of evidence to improve the 
reliability of instrument such as testing and 
retesting the survey and recruiting a larger 
more representative national sample, especially 
with students from schools not extensively 
engaged in civic education and larger samples 
of students from various racial and ethnic 
groups; and 



www.civicyouth.org 



23 






CIRCLE Working Paper 47: June 2006 Developing Indicators and Measures of Civic Outcomes for Elementary School Students 



• the development of curriculum materials for 
teaching civic education at the elementary 
level that is consistent with the conceptual 
framework described in this study. 

In many ways, the elementary level is 
an ideal time to create a strong and meaningful 
foundation for the civic knowledge, skills, and 
dispositions needed to prepare and engage 
students as active citizens through a more robust, 
comprehensive developmental framework for 
citizenship education that begins with younger 
ages and addresses civic skills and dispositions to 
the same degree as civic knowledge. Similarly, 
driven by our needs to assess the work of our 
school, there is a need for greater attention to 
age-appropriate, instrument identification and 
development for elementary aged students to 
document student civic development by focusing on 
what they can do. Addressing this need will also 
assist other public elementary schools interested in 
recapturing their civic mission and in creating a K- 
12 developmental framework for civic development. 
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Appendix A: Student Survey Used in National Pilot 



Dear Student: 



Student Survey 
Fall Semester, 2005 



By taking this survey, you are helping to create a survey that will help us understand what students think 
about their schools and communities. Please try to answer these questions as honestly and accurately 
as possible. It is important that I understand how you think and feel about these issues. 

If there are words you do not understand, please CIRCLE them. 

If there are questions you do not understand, please CIRCLE the number of the question. 

Everything you say will be kept private so please do not put your name on this form. But please complete 
the Student Information section below to help us match your responses at the end of the year. Thank you 
so much for your time and effort. 



Sincerely, 

Bernadette Chi 

East Bay Conservation Corps Charter School, Oakland, California 



STUDENT INFORMATION 

1. Today’s Date: 

2. Your School: Teacher: Grade: 

3. Gender: □ Boy □ Girl 

4. When is your birthday? (Month/Day /Year) 

5. What is your middle initial? (If you do not have a middle name, please put “Z”.) 

6. What is your ethnic/racial group? (Optional) 

Q European/white O Asian/Pacific Islander 

Q Latino/Hispanic O American Indian 

I I African American O Other (or more than one ethnic/racial 

group) 



Student Survey - Page 1 
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PART I: Please circle the number that shows what you think about each statement. 



Example: 

I like ice cream. 



No Way! Not really Sort of 

1 2 3 




No Way! Not Really 


Sort of 


Yes! 


1. If I break something, I try to fix it. 1 


2 


3 


4 


2. To solve most problems, I have to learn how to work with others. 1 


2 


3 


4 


3. I try to think how someone else would feel before I say something. 1 


2 


3 


4 


4. I want to help when I see someone in need. 1 


2 


3 


4 


5. It is important for me to get information to support my opinions. 1 


2 


3 


4 


6. I try to get my family to recycle at home. 1 


2 


3 


4 


7. Once I know what needs to be done, I am good at planning how to 1 

do it. 


2 


3 


4 


8. If friends are fighting, I try to get them to talk to each other and 1 

stop fighting. 


2 


3 


4 


9. I try to think before I say something. 1 


2 


3 


4 


10. When I make a decision I try to think about how other people will 1 
be affected. 


2 


3 


4 


11. 1 try to be kind to other people. 1 


2 


3 


4 


12. 1 usually do what I am supposed to do. 1 


2 


3 


4 


13. When I see something that needs to be done, I try to get my friends 1 
to work on it with me. 


2 


3 


4 


14. 1 want to have friends who have different backgrounds from me. 1 


2 


3 


4 


15. Most students at this school treat each other with respect. 1 


2 


3 


4 


16. 1 can learn more from working on group projects than from 1 

working alone. 


2 


3 


4 


17. 1 have a responsibility to help keep the community clean. 1 


2 


3 


4 


18. 1 am pretty good at organizing a team of kids to do a project. 1 


2 


3 


4 


19. It is important for me to follow the rules even if no one is 1 

watching. 


2 


3 


4 


20. 1 apologize when I hurt someone’s feelings. 1 


2 


3 


4 
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PAGE 2: Please circle the number that shows what you think about each statement. 

No Way! Not Really Sort of Yes! 



21. If I’m the leader of a group, I make sure that everyone in the group 12 3 4 

feels important. 




42. 1 believe that I can make a difference in my community. 12 3 4 
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PAGE 3: Please circle the number that shows what you think about each statement. 

No Way! Not Really Sort of Yes! 



43.1 can learn a lot from people with backgrounds and experiences 12 3 4 

that are different from mine. 



44. When 1 play with others, I take turns. ] 


l 2 3 


45. 1 know how to avoid a fight when I need to. ] 


l 2 3 


46. 1 want to help when I see someone having a problem. ] 


l 2 3 


47. 1 do my part to help the environment. ] 


l 2 3 



48. 1 make sure I understand what another person is saying before I 1 

respond. 


2 


3 


4 


49. When I am listening to someone, 1 try to understand what they are 1 
feeling. 


2 


3 


4 


50. Students can talk to the teachers in this school about things that 1 

are bothering them. 


2 


3 


4 




PART II: Please answer the following questions by circling the best response (a. b.j:, or d) 

51. Which of these people make local laws? 



a. Police officers c. City council members 

b. Newspaper reporters d. Business leaders 

52. July 4 is a national holiday that celebrates the day when: 

a. the American colonies declared their c. women won the right to vote in elections, 

independence from Britain d. ^ le q' V ;7 War officially ended. 

b. the Pilgrims arrived in the New World. 



53. “I pledge allegiance to the flag of the United States of America, and to the republic for which it stands, one 
nation under God, indivisible, with liberty and justice for all.” 

What are you promising when you say the pledge of allegiance? 

a. I promise to be loyal to the ideals of the United c. I promise to salute the flag when I see it in 

States. parades. 

b. I promise to study the laws of the United States. d. I promise to join the army of the United States. 



54. A student notices that the playing fields at her school have become littered with trash. How could she best 

show her civic responsibility? 

a. Complain to her friends that no one at the school c. Organize her class or school club to 

cares about the way it looks. clean up the playing fields. 

b. Ignore the mess and take her friends to play in an d. Call the police to report that the 

area that is still clean school ’s fields are dirty. 

55. What is a good reason to have laws? 

a. To get people into trouble. c. To protect people. 

b. To help the president make money. d. To keep police busy. 



© Thank you for your best effort!! © 
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Appendix B: Student Observation Checklists for Grades KJ1, 2/5 and 4/5 

Student Observation Checklist of Civic Skills and Behaviors 
For Kindergarten and First Grades 

Directions: The intent of this assessment is to assist you in documenting student civic development 
during the school year and to provide evidence for your report card ratings on student civic literacy. 

Please respond to the following question for each of the items below, using a scale of 1 to 4: 



How frequently does the student exhibit the described skill or behavior? 

1 = Almost never 2 = Sometimes 3 = Frequently 4 = Almost always 





Start 

of 

Year 


1st 

Report 


2nd 

Report 


End 

of 

Year 


PERSONAL RESPONSIBILITY 




Personally Responsible Behavior 










Makes responsible choices during independent work or free time 










Raises hand before speaking 










Shows impulse control 










Uses materials respectfully 










Performs class job 










Picks up belongings 










Productive Work Habits 










Makes efficient use of class time 










Participates in class and actively engages in the material 










Works independently with minimum teacher support by attempting to resolve 
questions before seeking help 










Completes class assignments 










Works with care and shows best effort 










CARING FOR OTHERS AND FOR COMMUNITY 




Caring for Others 










Shares materials with others 










Is sympathetic toward others’ pain or struggles 










Voluntarily helps peers who require it 










Does not tease others based on characteristics that are different from him/her 
(gender, race, class, disability, etc.) 










Caring for Group or Community 










Demonstrates concern for needs in the class or community. 










Is cooperative in group situations with peers 










Puts trash and recycling items in appropriate receptacles 










Does not waste resources/materials 











(continued on next page) 
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How frequently does the student exhibit the described skill or behavior? 

1 = Almost never 2 = Sometimes 3 = Frequently 4 = Almost always 



Start 






End 


of 


1st 


2nd 


of 


Year 


Report 


Report 


Year 



LEADERSHIP 


Is able to help make decisions that benefit the group 










Leads by setting a good example for others 










Takes care of others 










Perspective Taking Skills 










Changes behavior if it negatively affects others 










Critical Thinking Skills 










Is able to give reasons for his/her opinions 










Brainstorms different solutions to problems 










Communication Skills 










Listens while others are speaking (e.g., as in circle or sharing time) 










Expresses needs and feelings in a constructive manner 










Shows appropriate body language to demonstrate active listening (e.g. leans 
toward the speaker, faces the speaker, nods head, etc.) 










Confidently and clearly expresses her/his thoughts in front of a group 










Group Membership Skills 










Can accept not getting her/his own way 










Cooperates with peers in group activities 










Takes turns when in groups 










Can work in pairs or small groups 










Conflict Resolution Skills 










Appropriately copes with aggression from others (e.g., tries to avoid a fight, 
walks away, seeks assistance, defends self when necessary) 










Expresses emotions appropriately (e.g. without becoming violent or shutting 
down for long periods of time) 










Uses “I” messages or other respectful communication to resolve problems 











www.civicyouth.org 



33 







CIRCLE Working Paper 47: June 2006 Developing Indicators and Measures of Civic Outcomes for Elementary School Students 



Student Observation Checklist of Civic Skills and Behaviors 
For Second and Third Grades 

Directions: The intent of this assessment is to assist you in documenting student civic development 
during the school year and to provide evidence for your report card ratings on student civic literacy. 

Please respond to the following question for each of the items below, using a scale of 1 to 4: 

How frequently does the student exhibit the described skill or behavior? 

1 = Almost never 2 = Sometimes 3 = Frequently 4 — Almost Always 





Start 

of 

Year 


1st 

Prog 

Report 


2nd 

Prog 

Report 


End 

of 

Year 


PERSONAL RESPONSIBILITY 






Personally Responsible Behavior 










Makes constructive choices during free time 










Raises hand before speaking 










Picks up belongings 










Shows impulse control 










Follows through on responsibilities such as class job 










Productive Work Habits 










Completes class and homework assignments 










Is organized (e.g. locates materials quickly and starts work) 










Participates in class and actively engages in the material (e.g. asks clarifying 
questions) 










Works independently with minimum teacher support by attempting to resolve 
questions before seeking help 










Listens carefully to teacher instructions and direction for assignments 










Stays on task and shows best effort 










CARING FOR OTHERS AND FOR COMMUNITY 






Caring for Others 










Voluntarily helps peers who require it (e.g. shares materials, etc.) 










Is concerned about the feelings of others (e.g. asks about a student who is upset or 
includes a student who is left out) 










Actively respects others in actions and words 










Shows patience with younger children 










Caring for Group or Community 










Demonstrates concern for needs in the class or community. 










Is cooperative in group situations with peers 










Is able to see the needs of the group as important as one's own (e.g. waits turn, 
accepts not getting own way if group decides differently, etc.) 










Does not single out or tease others based on characteristics (e.g. gender, race, 
class, neighborhood, disability, etc.) 










(continued on next page) 
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How frequently does the student exhibit the described skill or behavior? 

/ = Almost never 2 = Sometimes 3 = Frequently 4 = Almost Always 






Start 

of 

Year 


1st 

Prog 

Report 


2nd 

Prog 

Report 


End 

of 

Year 


Caring for Group or Community (continued) 










Is able to work on a project with a person who is different from him/her 










Puts trash and recycling items in appropriate receptacles 










Encourages other students to recycle or to clean up 










Does not waste resources/materials 










LEADERSHIP 


Assumes leadership role in peer activities 










Is able to help make decisions that benefit the group 










Advocates for changes or improvements 










Makes choices to do what is right, even when peers make other choices 










Perspective Taking Skills 










Shows understanding for other people’s feelings, ideas or actions 










Is able to interpret what peers are trying to do and understand their intentions 










Is aware of the effect of his/her behavior on others 










Critical Thinking Skills 










Is able to give reasons for his/her opinions 










Is able to separate facts from opinions 










Demonstrates problem solving skills (e.g. brainstorms multiple solutions to a 
problem, breaks tasks into simpler activities, etc.) 










Communication Skills 










Expresses needs and feelings in a constructive manner 










Listens to the ideas of others even if s/he disagrees with them 










Can succinctly present main idea or point 










Asks questions that extends what is being discussed 










Shows appropriate body language to demonstrate active listening (e.g. leans 
toward the speaker, faces the speaker, nods head, etc.) 










Confidently and clearly expresses her/his thoughts in front of a group 










Group Membership Skills 










Compromises with peers when situation calls for it 










Works with others to solve a problem 










Is willing to wait his/her turn 










Conflict Resolution Skills 










Appropriately copes with aggression from others (e.g., tries to avoid a fight, walks 
away, seeks assistance, defends self when necessary) 










Expresses emotions appropriately (e.g. without becoming violent or shutting down 
for long periods of time) 










Uses “I” messages or other respectful communication to resolve problems 










Tries to use own resources first and then seeks mediator if needed 
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Student Observation Checklist of Civic Skills and Behaviors 
For Fourth and Fifth Grades 

Directions: The intent of this assessment is to assist you in documenting student civic development during the 
school year and to provide evidence for your report card ratings on student civic literacy. 

Please respond to the following question for each of the items below, using a scale of 1 to 4: 



Flow frequently does the student exhibit the described skill or behavior? 

1 = Almost never 2 = Sometimes 3 = Frequently 4 = Almost always 





Start 

of 

Year 


1st 

Prog 

Report 


2nd 

Prog 

Report 


End 

of 

Year 


PERSONAL RESPONSIBILITY 








Personally Responsible Behavior 










Makes responsible choices during free time 










Follows through on responsibilities such as class job 










Uses materials respectfully 










Shows impulse control 










Productive Work Habits 










Completes class and homework assignments 










Is organized (e.g. locates materials/items quickly and starts work) 










Participates in class and actively engages in the material 










Works independently with minimum teacher support by attempting to resolve 
questions before seeking help 










Stays on task and shows best effort 










CARING FOR OTHERS AND FOR COMMUNITY 








Caring for Others 










Voluntarily helps peers who require it (e.g. shares materials, etc.) 










Is concerned about the feelings of others (e.g. asks about a student who is upset or 
includes a student who is left out) 










Shows patience with younger children 










Caring for Group or Community 










Demonstrates concern for needs in the class or community. 










Is cooperative in group situations with peers 










Is able to see the needs of the group as important as one's own (e.g. waits turn, 
accepts not getting own way if group decides differently, etc.) 










Does not single out or tease others based on characteristics (e.g. gender, race, class, 
neighborhood, disability, etc.) 










Is able to work on a project with a person who is different from him/her 










Puts trash and recycling items in appropriate receptacles 










Considers effect of behavior/choices on the environment 










Does not waste resources/materials 
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How frequently does the student exhibit the described skill or behavior? 

1 = Almost never 2 = Sometimes 3 = Frequently 4 = Almost always 



Start 


1st 


2nd 


End 


of 


Prog 


Prog 


of 


Year 


Report 


Report 


Year 



LEADERSHIP 


Assumes leadership role in peer activities 










Is able to help make decisions that benefit the group 










Advocates for changes or improvements 










Makes choices to do what is right, even when peers make other choices 










Perspective Taking Skills 










Shows understanding for other people’s feelings, ideas or actions 










Is able to interpret what peers are trying to do and understand their intentions 










Is aware of the effect of his/her behavior on others 










Critical Thinking Skills 










Is able to give reasons for his/her opinions 










Is able to separate facts from opinions 










Demonstrates problem solving skills (e.g. brainstorms multiple solutions to a 
problem, breaks tasks into simpler activities, etc.) 










Communication Skills 










Expresses needs and feelings in a constructive manner 










Listens to the ideas of others even if s/he disagrees with them 










Can succinctly present main idea or point 










Asks questions that extends what is being discussed 










Shows appropriate body language to demonstrate active listening (e.g. leans toward 
the speaker, faces the speaker, nods head, etc.) 










Confidently and clearly expresses her/his thoughts in front of a group 










Group Membership Skills 










Compromises with peers when situation calls for it 










Works with others to solve a problem 










Is willing to wait his/her turn 










Conflict Resolution Skills 










Appropriately copes with aggression from others (e.g., tries to avoid a fight, walks 
away, seeks assistance, defends self when necessary) 










Expresses emotions appropriately (e.g. without becoming violent or shutting down 
for long periods of time ) 










Uses “I” messages or other respectful communication to resolve problems 










Tries to use own resources first and then seeks mediator if needed 
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Appendix C. Pearson Correlations between Scales 





Personal 

Respons. 


Concern for 
others 


Value of 
Group 
Work 


Caring for 
Community 


Appreciate. 

Diversity 


Env. 

Steward. 


Leadership 

Efficacy 


Critical 

Thinking 


Civic Part. 
Skills 


School as 
Dem. 
Comm. 


Personal 

Responsibility 


i 


,599(**) 


,296(**) 


.507(**) 


,265(**) 


.41 3(**) 


,579(**) 


.346(**) 


,653(**) 


,428(**) 


Concern for Others 


,599(**) 


1 


■253(**) 


,532(**) 


,264(**) 


■520(**) 


■561 n 


,392(**) 


.684(**) 


■367 (**) 


Value of Group 
Work 


,296(**) 


.253(**) 


1 


,292(**) 


. 1 66(**) 


,248(**) 


,352(**) 


,148(**) 


.352C) 


,300(**) 


Caring for 
Community 


,507(**) 


,532(**) 


,292(**) 


1 


.261 n 


,640(**) 


■571 (**) 


,339(**) 


,556(**) 


.405(**) 


Appreciating 

Diversity 


,265(**) 


,264(**) 


. 1 66(**) 


.261 (**) 


1 


,286(**) 


.331 (**) 


,283(**) 


,366(**) 


■192(**) 


Environmental 

Stewardship 


.41 3(**) 


■520 (**) 


,248(**) 


,640(**) 


,286(**) 


1 


.467(**) 


.31 2(**) 


.487(**) 


.356(**) 


Leadership Efficacy 


,579(**) 


■561 (**) 


.352(**) 


.571 (**) 


.331 (**) 


■467 (**) 


1 


■41 4(**) 


■61 3(**) 


■34i n 


Critical Thinking 


,346(**) 


,392(**) 


,148(**) 


,339(**) 


,283(**) 


■31 2(**) 


,414(**) 


1 


,449(**) 


■250(**) 


Civic Participation 
Skills 


,653(**) 


,684(**) 


,352(**) 


.556(**) 


,366(**) 


,487(**) 


■61 3(**) 


449(**) 


1 


,448(**) 


School as 
Community 


,428(**) 


,367(**) 


,300(**) 


,405(**) 


■192(**) 


,356(**) 


■341(**) 


■250(**) 


,448(**) 


1 



** Correlation is significant at the 0.01 level (2-tailed). 
* Correlation is significant at the 0.05 level (2-tailed). 
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Appendix D. Tests of Significance for Findings by Gender 





t 


df 


Mean 
Difference 
(Boys minus 
Girls) 


Std. Error 
Difference 


Personal Responsibility** 


Equal variances 
assumed 


-4.175 


496 


-.16574 


.03970 


Concern for others** 


Equal variances 
assumed 


-6.772 


493 


-.29537 


.04362 


Value of Group Work* 


Equal variances 
assumed 


-2.136 


519 


-.14839 


.06948 


Caring for Community** 


Equal variances 
not assumed 


-3.616 


435.222 


-.21085 


.05832 


Appreciating Diversity 


Equal variances 
not assumed 


-1.634 


420.187 


-.10871 


.06654 


Environmental 

Stewardship* 


Equal variances 
not assumed 


-2.266 


449.083 


-.14601 


.06444 


Leadership Efficacy** 


Equal variances 
assumed 


-3.804 


486 


-.17622 


.04632 


Critical Thinking* 


Equal variances 
not assumed 


-2.219 


442.084 


-.10886 


.04905 


Civic Participation Skills** 


Equal variances 
assumed 


-6.141 


479 


-.29749 


.04845 


School as Community 


Equal variances 
not assumed 


-.925 


433.057 


-.04158 


.04494 


Civic Knowledge 


Equal variances 
not assumed 


-1.394 


419.244 


-.02993 


.02148 



* Indicates significance at <.05 level 

* indicates significance at <.01 level 
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Appendix E. Tests of Significance for Findings by White and Non-White Subpopulations 





t 


df 


Mean Difference 
(White versus 
Non-Whites) 


Std. Error 
Difference 


Personal Responsibility 


Equal variances 
assumed 


-.295 


331 


-.01479 


.05008 


Concern for Others* 


Equal variances 
not assumed 


2.166 


322.371 


.12170 


.05619 


Value of Group Work* 


Equal variances 
not assumed 


-2.228 


310.895 


-.18569 


.08334 


Caring for Community 


Equal variances 
not assumed 


.747 


314.671 


.05436 


.07275 


Appreciating 

Diversity 


Equal variances 
assumed 


1.770 


323 


.13760 


.07773 


Environmental 

Stewardship 


Equal variances 
not assumed 


-.528 


322.835 


-.04159 


.07877 


Leadership Efficacy 


Equal variances 
not assumed 


.241 


317.580 


.01362 


.05649 


Critical Thinking 


Equal variances 
not assumed 


.791 


325.795 


.04577 


.05786 


Civic Participation Skills 


Equal variances 
not assumed 


-.696 


308.617 


-.04300 


.06178 


School as Community 


Equal variances 
not assumed 


-.614 


281.535 


-.03326 


.05416 


Civic Knowledge** 


Equal variances 
not assumed 


2.962 


314.224 


.07571 


.02556 



* Indicates significance at <.05 level 

* indicates significance at <.01 level 



www.civicyouth.org 



40 







CIRCLE Working Paper 47: June 2006 Developing Indicators and Measures of Civic Outcomes for Elementary School Students 



Appendix F. Descriptive Statistics of Full National Sample. 



Scale 


N 


Minimum 


Maximum 


Mean 


Std. 

Deviation 


Personal 

Responsibility 


503 


1.14 


4.00 


3.4913 


.44330 


Concern for others 


500 


1.00 


4.00 


3.5050 


.49989 


Value of Group Work 


526 


1.00 


4.00 


3.3484 


.78418 


Caring for Community 


504 


1.00 


4.00 


3.0920 


.64916 


Appreciating Diversity 


496 


1.00 


4.00 


3.2823 


.71825 


Environmental 

Stewardship 


510 


1.00 


4.00 


2.8375 


.71615 


Leadership Efficacy 


493 


1.17 


4.00 


3.2130 


.51194 


Critical Thinking 
Skills 


507 


1.33 


4.00 


3.3902 


.53950 


Civic Participation 
Skills 


486 


1.00 


4.00 


3.3182 


.54221 


School as Community 


477 


1.00 


4.00 


3.3566 


.48081 


Civic Knowledge 


476 


.00 


1.00 


.7462 


.22879 
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CIRCLE (The Center for Information and Research on Civic Learning and Engagement) promotes research 
on the civic and political engagement of Americans between the ages of 15 and 25. Although CIRCLE 
conducts and funds research, not practice, the projects that we support have practical implications 
for those who work to increase young people's engagement in politics and civic life. CIRCLE is also a 
clearinghouse for relevant information and scholarship. CIRCLE was founded in 2001 with a generous 
grant from The Pew Charitable Trusts and is now also funded by Carnegie Corporation of New York. It is 
based in the University of Maryland's School of Public Policy. 






CIRCLE 

The Center for Information & Research 
on Civic Learning & Engagement 
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